FREE EXPONENTIAL FAMILIES AS KERNEL FAMILIES

Gradient-free Hamiltonian Monte Carlo with Efficient Kernel Exponential Families

We propose Kernel Hamiltonian Monte Carlo (KMC), a gradient-free adaptive MCMC algorithm based on Hamiltonian Monte Carlo (HMC). On target densities where classical HMC is not an option due to intractable gradients, KMC adaptively learns the target’s gradient structure by fitting an exponential family model in a Reproducing Kernel Hilbert Space. Computational costs are reduced by two novel effi...

Exponential families

Optimally approximating exponential families

This article studies exponential families E on finite sets such that the information divergence D(P E) of an arbitrary probability distribution from E is bounded by some constant D > 0. A particular class of low-dimensional exponential families that have low values of D can be obtained from partitions of the state space. The main results concern optimality properties of these partition exponent...

Asymptotics of Exponential Families

if we use the convention that additive terms that do not contain the parameter may be dropped from log likelihoods (because they do not affect any likelihood-based statistical inferences). Thus we have another definition of exponential families: a statistical model is an exponential family of distributions if there exists a statistic Y and a parameter θ such that the log likelihood has the form...

Satisfiability with Exponential Families

Fix a set S ⊆ {0, 1}∗ of exponential size, e.g. |S ∩ {0, 1}n| ∈ Ω(αn), α > 1. The S-SAT problem asks whether a propositional formula F over variables v1, . . . , vn has a satisfying assignment (v1, . . . , vn) ∈ {0, 1}n ∩S. Our interest is in determining the complexity of S-SAT. We prove that S-SAT is NP-complete for all context-free sets S. Furthermore, we show that if S-SAT is in P for some e...